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APPLICATION  OF  THE 

RISK  METHODOLOGY 

USED  FOR  MEASURING  LICENSED  AND  OUT-OF- 
PROVINCE  DRINKING  WATER  TESTING  LABORATORY 
INSPECTION  RESULTS 


The  Ministry  of  the  Environment  (MOE)'s  drinking 
water  testing  laboratory  inspection  program  is 
an  important  aspect  of  our  drinking  water  safety 
net.  The  ministry  and  its  partners  share  a  common 
commitment  to  excellence  and  we  continue  to 
work  toward  the  goal  of  100  per  cent  regulatory 
compliance. 

The  ministry  has  a  rigorous  and  comprehensive 
inspection  program  for  licensed  and  out-of- 
province  laboratories  that  analyze  Ontario  drinking 
water.  The  program  objective  is  to  determine  the 
compliance  of  these  laboratories  with  requirements 
under  the  Safe  Drinking  Water  Act  (SDWA)  and 
associated  regulations. 

Ontario. ca/drinkingwater 


It  is  the  responsibility  of  the  licensed  or  out-of- 
province  drinking  water  laboratories  to  ensure 
that  they  are  in  compliance  with  all  applicable 
legal  requirements. 

This  document  describes  the  new  risk  rating 
methodology,  which  will  be  applied  to  the  findings 
of  MOE  drinking  water  laboratory  inspection 
results  commencing  April  1,  2012.  The  primary 
goal  of  this  assessment  is  to  encourage  on-going 
improvement  of  compliant  laboratories. 

The  MOE  will  review  the  risk  rating  methodology 
every  three  years. 
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The  Ministry's  Drinking  Water  Laboratory  Inspection 
Protocol  contains  up  to  12  inspection  modules 
and  consists  of  approximately  370  regulatory 
questions.  Those  protocol  questions  are  also  linked 
to  definitive  guidance  that  ministry  inspectors 
use  when  conducting  drinking  water  laboratory 
inspections.  The  questions  address  a  wide  range  of 
regulatory  issues,  from  administrative  procedures 
to  adverse  water  quality  reporting.  Additionally, 
the  inspection  protocol  contains  a  number  of 
non-regulatory  questions.  These  best  practice 
questions  are  not  risk  rated.  Also,  questions  that 
are  part  of  the  "other  inspection  findings"  module 
are  currently  not  risk  rated.  It  is  possible  for  a 
laboratory  to  have  a  required  action  as  part  of  this 
module  with  no  change  to  their  risk  rating. 

A  team  of  laboratory  and  drinking  water  specialists 
in  the  ministry  have  assessed  each  of  the  inspection 
protocol  regulatory  questions  to  determine  the 
assigned  risk  of  the  laboratory  not  complying  with 
each  regulatory  requirement.  This  assessment  is 
based  on  established  provincial  risk  assessment 
principles,  with  each  question  receiving  a  risk  rating 
referred  to  as  the  Question  Risk  Rating.  Based  on 
the  number  of  areas  where  a  laboratory  is  deemed 
to  be  non-compliant  during  the  inspection,  and  the 
significance  of  these  areas  to  administrative  and 
health  consequences,  a  risk-based  inspection  rating 
is  calculated  by  the  ministry  for  each  drinking  water 
laboratory. 

It  is  important  to  note  that  an  inspection  rating 
that  is  less  than  100  per  cent  does  not  necessarily 
mean  that  the  laboratory  is  not  analyzing  drinking 
water  improperly  or  not  reporting  data  as  required. 
It  provides  laboratories  with  information  they 
can  use  to  make  better  decisions  and  improve 
laboratory  compliance.  To  that  end,  the  ministry 


will  continue  to  work  with  the  laboratories  to 
ensure  they  know  what  to  do  to  achieve  full 
compliance. 

The  new  inspection  rating  refiects  the  inspection 
results  of  the  drinking  water  laboratory  for  each 
inspection  cycle.  At  least  two  inspection  cycles 
are  carried  out  per  year.  Since  different  inspection 
modules  may  be  employed  for  every  inspection 
cycle,  the  inspection  ratings  are  not  comparable 
year-to-year.  The  rating  record  does  encourage 
continuous  improvement  and  allows  laboratories 
to  identify  specific  areas  requiring  attention. 

Determining  the  Risk  Rating 
for  Drinking  Water  Testing 
Laboratories 

The  risk  management  approach  used  for  licensed 
and  out-of-province  drinking  water  laboratories 
is  aligned  with  the  Government  of  Ontario's  Risk 
Management  Framework.  Risk  management  is 
a  systematic  approach  to  identifying  potential 
hazards;  understanding  the  likelihood  and  conse- 
quences of  the  hazards;  and  taking  steps  to  miti- 
gate the  risks  if  necessary  and  as  appropriate. 

The  Risk  Management  Framework  provides  a 
formula  to  be  used  in  the  determination  of  risk: 

RISK  =  LIKELIHOOD  x  CONSEQUENCE 

(of  the  consequence) 

Every  regulatory  question  in  the  inspection 
protocol  possesses  a  likelihood  value  (L)  for  an 
assigned  consequence  value  (C)  as  described  in 
Table  1  and  Table  2. 
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TABLE  1: 

Likelihood  of  Consequence  Occurring 

Lilceliliood  Value 

0%  -  0.99%  (Possible  but  Highly  Unlikely) 

L  =  0 

1  -10%(Unlil<ely) 

L  =  1 

1 1  -  49%  (Possible) 

L  =  2 

50  -  89%  (Likely) 

L  =  3 

90-100%  (Almost  Certain) 

L  =  4 

TABLE  2: 

Consequence 

Consequence  Value 

Minor  Administrative  Consequence 

C  =  1 

Medium  Administrative  Consequence 

C  =  2 

Major  Administrative  Consequence 

C  =  3 

Minor  Health  Consequence 

C  =  4 

Medium  Health  Consequence 

C  =  5 

Major  Health  Consequence 

C  =  6 

The  consequence  values  (1  through  6)  are  select- 
ed to  align  with  other  risk-based  programs  and 
projects  currently  under  development  or  in  use 
within  the  ministry  as  outlined  in  Table  2. 

The  Question  Risk  Rating  for  each  regulatory  in- 
spection question  is  derived  from  an  evaluation  of 
every  identified  consequence  and  its  corresponding 
likelihood  of  occurrence: 

•  All  levels  of  consequence  are  evaluated  for 
their  potential  to  occur 

•  Greatest  of  all  the  combinations  is  selected. 


The  Question  Risk  Rating  quantifies  the  risk  of 
non-compliance  of  each  question  relative  to  the 
others.  Questions  with  higher  values  are  those 
with  a  potentially  more  significant  impact  on 
drinking  water  safety  and  a  higher  likelihood  of 
the  consequence  occurring.  The  highest  possible 
value  would  be  24  (4x6)  and  the  lowest  would  be 
0  (0x1). 

Table  3  presents  a  sample  question  showing  the 
risk  rating  determination  process. 


TABLE  3: 


Does  the  laboratory  have  an  authorized  adverse  reporting  Policy/Procedure? 

Risk  =  Likelihood  x  Consequence 

C=1 

C=2 

C=3 

C=4 

C=5 

C=6 

IVIinor 

Administrative 
Consequence 

IVIedium 

Administrative 
Consequence 

IVIajor 

Administrative 
Consequence 

Minor 

Health 
Consequence 

Medium 

Health 
Consequence 

Major 

Health 
Consequence 

L=4 
(Almost  Certain) 

L=4 
(Almost  Certain) 

L  =  1 
(Unlikely) 

L=4 
(Almost  Certain) 

L=2 
(Possible) 

L=1 
(Unlikely) 

R  =  4 

R  =  8 

R  =  3 

R  =  16 

R  =  10 

R  =  6 
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Application  of  the 
iVIethodoiogy  to  Laboratory 
Inspection  Results 

Based  on  the  results  of  a  drinking  water  laboratory 
inspection,  an  overall  inspection  risk  rating 
is  calculated.  During  an  inspection,  inspectors 
answer  the  questions  that  relate  to  regulatory 
compliance  and  input  their  responses  as  "yes," 
"no"  or  "not  applicable"  into  the  Ministry's 
Laboratory  and  Waterworks  Inspection  System 
(LWIS)  database.  A  "no"  response  indicates  non- 
compliance. The  maximum  number  of  regulatory 
questions  asked  by  an  inspector  varies  by: 
laboratory  (i.e.,  licensed,  unlicensed)  and  type  of 
inspection  (i.e.,  annual  1,  annual  2,  focused,  etc.) 

The  risk  ratings  of  all  non-compliant  answers 
are  summed  and  divided  by  the  sum  of  the  risk 
ratings  of  all  questions  asked  (maximum  question 
rating).  The  resulting  inspection  risk  rating  (as 
a  percentage)  is  subtracted  from  100  per  cent  to 
arrive  at  the  final  inspection  rating. 


Reporting  Results  to  Drinking 
Water  Testing  Laboratories 

A  summary  of  inspection  findings  for  each 
laboratory  is  generated  in  the  form  of  an 
Inspection  Rating  Record  (IRR).  The  findings 
are  grouped  into  the  12  possible  modules  of  the 
inspection  protocol,  which  would  provide  the 
laboratory  with  information  on  the  areas  where 
they  need  to  improve. 


The  12  modules  are: 


1.  General 

2.  Chain  of 
Custody 


8.  Sample  Handling 

9.  Subcontracting 

10.  Methods 


3.  Data  Retention 

11.  Drinking 

4.  Drop-off  Depots  Water  Testing 

Compliance 


5.  DWIS 

6.  Projects 

7.  Reporting 


12.  Other  Inspection 
Findings 


The  laboratory  will  receive  an  IRR  with  each 
inspection  report.  The  IRR  is  confidential  to  each 
laboratory  and  is  not  publicly  reported.  However, 
cumulative  results  will  be  part  of  the  annual 
report  on  drinking  water  by  the  Chief  Drinking 
Water  Inspector. 


For  further  information,  please  contact: 

Laboratory  Licensing  and 
Compliance  Supervisor 

IVIinistry  of  the  Environment 
125  Resources  Road, 
Toronto,  ON  M9P  3V6 

Phone:  416-235-6517 
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